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Post  Hospitalization  Outcomes  Studies 
University  of  Minnesota 

Final  Report 

The  Health  Care  Financing  Administration  (HCFA),  working  in  coordination  with  the  Agency 
for  Health  Care  Policy  and  Research  (AHCPR),  contracted  with  the  School  of  Public  Health  at  the 
University  of  Minnesota  to  conduct  the  Post  Hospitalization  Outcomes  Studies  (PHOS).  The 
contract  called  for  the  development  of  a  method  for  assessing  patient  outcomes  for  Medicare 
beneficiaries  following  discharge  from  an  acute  hospital  stay  based  upon  the  patients'  condition  or 
on  the  type  of  medical  procedures  performed  and  linking  this  information  to  each  patient  record  in 
the  Medicare  data  base.  Information  from  this  research  will  be  used  to  establish  a  methodological 
framework  for  future  outcomes  research.  In  addition,  a  retrospective  by-product  of  this  study  may 
be  a  model  for  data  collection  on  both  processes  and  outcomes  of  care  that  could  be  routinely  used 
as  part  of  a  quality  assurance  program  that  compares  the  case-mix  adjusted  outcomes  of  care 
among  hospitals. 

The  primary  objective  of  PHOS  was  to  develop  an  approach  to  collect  information  on 
Medicare  enroUees  regarding  important  patient  outcomes  at  relevant  time  periods  after  hospital 
discharge.  Patient  outcomes  of  hospital  care  are  influenced  by  at  least  four  major  categories  of 
variables:  patient's  baseline  status  before  the  incident  that  led  to  the  hospitalization  (which 
determines  the  likely  ceiling  on  improvement),  patient  characteristics  (both  clinical  and  socio- 
demographic),  specifics  of  the  clinical  intervention  during  the  hospitalization,  and  formal  and 
informal  care  received  after  hospital  discharge.  Information  on  patient  demographic  characteristics, 
outcomes,  and  care  arrangements  were  collected  for  this  project  by  telephone  interviews  with  the 
patient  at  the  time  intervals  after  discharge:  within  two  months  (when  baseline  status  was 
determined  retrospectively),  between  four  and  six  months  (to  assess  intermediate  outcomes),  and  at 
one  year  post  discharge.  Clinical  information  was  abstracted  from  the  patient's  hospital  medical 
record  and  information  on  service  utilization  and  costs  was  abstracted  from  Medicare  data  files. 
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The  combination  of  clinical  and  personal  patient  characteristics  allows  for  the  description  of 
national  patterns  of  outcomes  adjusted  for  case  mix. 

Data  from  PHOS  provides  a  more  comprehensive  picture  of  long-term  outcomes,  controlling 
for  hospital  care,  by  monitoring  the  post-hospitalization  course  and  functional  status  of  Medicare 
beneficiaries.  Understanding  the  factors  that  influence  patients'  outcomes  has  the  potential  for 
improving  the  quality  of  patient  care  by  increasing  the  clinicians'  ability  to  develop  appropriate 
treatment  plans  for  patients  with  medical  conditions.  Specifically,  the  goals  of  these  studies  were  to: 

•  determine  the  types  and  rates  of  both  positive  outcomes  and  complications  or  untoward 
events  in  the  hospital  and  post  hospital  periods; 

•  determine  the  relationship  between  particular  types  of  service  or  combinations  of  service  and 
good  or  poor  outcomes; 

•  determine  the  impact  of  the  hospitalization  or  specific  procedures  on  the  subsequent 
progression  of  illness  and  the  changes  in  health  and  functional  status  (how  long  are  achieved 
benefits  maintained); 

•  investigate  the  relationship  between  the  patient's  satisfaction  with  the  hospitalization  and  . 
intermediate  and  long-term  outcomes;  and 

•  identify  sub-groups  of  patients  or  specific  patient  characteristics  that  identify  those  most 
likely  to  benefit  or  have  poor  outcomes  from  the  hospitalization. 

PHOS  developed  and  tested  a  method  for  collecting  information  on  outcomes  of  Medicare 
patients  for  selected  conditions  on  an  ongoing  basis.  The  research  endeavor  was  divided  into  a 
design  phase  and  an  implementation  phase.  Activities  in  each  phase  included: 

Design  Phase 

•  medical  conditions  to  be  studied  were  selected 

•  data  elements  and  sources  of  information  were  identified 

•  survey  instruments  were  designed  for  gathering  information  on  the  health  status  of  Medicare 
beneficiaries  prior  to  and  following  an  acute  care  hospitalization 
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•  information  to  be  collected  from  the  hospital  medical  records  was  identified 

•  information  to  be  collected  from  Medicare  administrative  data  was  identified 

•  sampling  frame  was  developed 

Implementation  Phase 

•  survey  data  collection  methodology  was  tested  and  significantly  refined  (pretest) 

•  data  was  collected  from  three  sources:  patients  (or  proxies),  hospital  medical  records, 
Medicare  data  files  (including  data  on  Medicare  enrollment,  service  utilization  and  costs  both 
prior  to  and  after  the  patient's  hospitalization) 

•  the  three  sources  of  data  were  merged 

This  merged  data  base  presents  a  comprehensive  picture  of  the  patients'  functional  and  clinical 
characteristics  and  patient  outcomes  post  discharge.  This  report  summarizes  the  activities  and 
results  of  both  the  design  and  implementation  phase  of  PHOS. 

Design  Phase 

Selection  of  Medical  Conditions  or  Procedures 

The  first  step  in  the  PHOS  design  phase  consisted  of  identifying  conditions  that  might  be 
studied.  Candidate  conditions  were  chosen  based  upon  the  opinion  of  a  Conditions  Identification 
Panel  (CIP)  composed  of  four  experts  (see  Appendix  A)  in  the  areas  of  health  status  measurement, 
severity  measures,  geriatrics,  and  epidemiology  of  aging. 

The  CIP  members  were  provided  with  a  structured  review  of  the  25  leading  candidate 
conditions  organized  according  to  the  following  background  information  (see  Appendix  B): 

•  incidence 

•  consequent  costs  and  speed  of  accruing  sufficient  number  of  cases 

•  risk  of  untoward  outcomes  and  associated  fiscal  consequences 

•  direct  and  indirect  costs 

•  intensity  and  duration  of  treatment 

3 


•  amount  and  extent  of  post-hospital  care 

•  data  on  risks,  course,  and  efficacy  of  interventions 

Based  upon  the  above  information,  the  CIP  members  recommended,  in  priority  order,  the 
following  ten  conditions  for  consideration  of  further  study: 

•  Stroke 

•  Hip  and  knee  replacement 

•  Pneumonia 

•  Cholecystectomy 

•  some  combination,  or  choice  of  one,  of  the  following: 
-Chronic  angina 

-CABG 

-Congestive  heart  failure 
-Acute  myocardial  infarction 

•  Urinary  Tract  Infection 

•  Large  and  small  bowel  procedures 

•  Hip  fracture 

Following  the  CIP  meeting,  the  PHOS  Advisory  Committee  (see  Appendix  C)  was  briefed  on 
the  outcomes  of  the  meeting.  The  list  of  ten  candidate  conditions  was  then  reduced  to  four  for 
survey  development:  Total  Hip  Replacement,  Cholecystectomy,  Large  Bowel  Procedures  for 
Cancer,  and  Pneumonia.  HCFA  would  undertake  surveys  of  two  conditions  initially  and  then 
determine  whether  to  undertake  the  additional  surveys  of  the  two  other  conditions. 

Criteria  used  in  selecting  the  four  conditions  for  further  study  included:  the  degree  to  which 
care  given  during  the  hospitalization  was  expected  to  influence  outcomes  (positive);  to  what  extent 
clear  treatment  choices  existed  (positive);  the  prevalence  of  the  condition  or  procedure  (positive); 
the  potential  overlap  with  the  topics  being  studied  by  the  PORTs  (negative);  and  maintaining  a 
balance  between  medical  and  surgical  conditions. 
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Stroke  was  eliminated  as  a  possible  condition  for  study  because  there  was  concern  that  the 
amount  of  post  hospital  care  commonly  provided  following  strokes  would  make  it  difficult  to 
attribute  outcomes  to  hospital  care.  This  judgment  might  be  different  now  in  light  of  active  interest 
in  the  role  of  thrombolytic  therapy.  Hip  procedures  were  seen  as  appropriate  for  further  study  but 
were  limited  to  elective  hip  replacements.  The  Advisory  Committee  felt  that  hip  fractures  and  hip 
replacements  fall  into  two  different  classes  because  they  follow  different  clinical  courses.  Elective 
hip  replacements  are  cleaner  cases  (have  less  complications  in  general)  to  study  and  because  they 
are  elective,  they  have  greater  policy  significance.  The  Advisory  Committee  also  cited  interest  in 
studying  the  different  approaches  used  in  elective  hip  replacement.  Cholecystectomy  was  selected 
for  further  study  because  the  procedures  used  to  treat  gall  bladder  problems  are  going  through  a 
major  transformation  with  the  introduction  of  laparoscopic  surgery.  The  timing  was  appropriate  to 
study  the  contrast  between  the  different  surgical  approaches.  Pneumonia  was  considered  an 
appropriate  condition  for  study  because  it  was  a  medical  condition  in  which  the  hospital 
intervention  would  have  a  significant  effect  on  outcomes.  The  various  cardiovascular  conditions 
were  discarded  because  of  the  overlap  with  work  being  conducted  by  the  PORTs.  Small  and  large 
bowel  procedures  were  included  for  study  because  of  the  interest  in  including  a  condition  that 
focused  on  patient  satisfaction.  Small  and  large  bowel  procedures  would  likely  include  colostomies 
for  cancer  and  other  conditions.  Outcomes  might  include  such  things  as  self-image,  social  activity, 
and  ability  to  manage  a  colostomy. 

Identification  of  Study  Conditions,  Data  Elements,  and  Sources  of  Information 

In  the  next  step  of  the  design  phase  of  PHOS,  Conditions  Specification  Panels  (CSP)  (see 
Appendix  D)  were  convened  for  each  of  the  conditions  to  be  studied:  Elective  Total  Hip 
Replacement,  Cholecystectomy,  Large  Bowel  Procedures  for  Cancer,  and  Pneumonia.  In  addition, 
a  General  Health  Outcomes  Panel  met  to  address  methodological  issues  around  collecting  outcomes 
data.  Networking  techniques  were  used  to  solicit  nominees  for  each  panel.  Panel  members  were 
selected  based  upon  their  clinical  expertise  and  availability.  Representatives  from  the  pneumonia 
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and  biliary  tract  PORTs  were  placed  on  the  relevant  CSP.  In  preparation  for  the  meetings  with  each 
CSP,  pertinent  literature  was  reviewed  and  provided  to  panel  members. 

During  the  CSP  meetings,  each  panel  was  asked  to:  1)  specify  case  definition;  2)  identify 
specific  outcome  measures;  3)  discuss  what  hospital  activities  influence  outcomes;  4)  recommend 
the  appropriate  intervals  and  length  of  PHOS  follow-up  for  patients  with  that  condition;  and  5) 
suggest  sources  of  data,  (e.g.,  patient  medical  record,  hospital  operating  log,  etc.).  Based  upon  the 
work  of  the  CSP  panels.  University  of  Minnesota  staff  developed  survey  instruments  and  chart 
abstraction  guidelines  to  be  used  to  gather  the  desired  baseline  data  and  data  about  outcomes  for 
each  of  the  four  conditions.  Following  development  of  initial  drafts  of  the  survey  instruments  for 
the  four  conditions,  the  CSPs  were  reconvened  to  obtain  their  input  on  the  designing  of  the 
instruments  and  the  elements  to  be  extracted  from  the  medical  record. 

Following  the  two  sets  of  CSP  meetings,  the  PHOS  Advisory  Committee  and  AHCPR  staff 
selected  two  conditions  for  pretesting:  cholecystectomy  and  elective  total  hip  replacement. 
Cholecystectomy  was  selected  because  it  was  a  common  procedure  in  which  the  technology  was 
changing  rapidly.  Although  performed  less  frequently,  elective  total  hip  procedures  were  selected 
because  there  are  well-defined  interventions  clearly  related  to  functional  sequelae.  While  both 
conditions  selected  are  surgical  procedures,  they  focus  on  different  surgical  specialties. 

Development  of  Survey  Instruments 

The  information  generated  from  the  CSP  members  was  incorporated  directly  into  the  survey 
instrument  development  process.  For  each  condition,  the  survey  instruments  included  questions 
regarding:  condition  specific  symptoms;  general  health;  functional  health;  mental  health;  cognitive 
ability;  living  arrangements;  and  use  of  post  acute  care  services.  Wherever  possible  existing 
questions  were  used  in  development  of  the  PHOS  survey  instruments.  Specifically,  general  health, 
mental  health,  and  physical  function  questions  came  from  the  SF  36,  which  was  used  in  the 
Medical  Outcomes  Study  (Stewart  et  al.,  1988).  Cognitive  questions  consisted  of  the  Mini  Mental 
Status  Questionnaire  (Folstein  et  al.,  1975).  Functional  status  and  post  acute  care  questions  came 
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from  the  survey  instruments  developed  for  the  Post  Acute  Care  Study  (Kane,  1994).  Condition 
specific  questions  were  derived  from  the  experts  on  the  CSPs.  Survey  instruments  for  each 
condition  were  revised  so  that  each  instrument  used  an  identical  set  of  generic  questions  linked  to 
condition-specific  questions  (see  Appendix  E).  Although  questionnaires  were  designed  for  the  four 
conditions  initially  selected  by  the  Advisory  Committee  and  discussed  by  the  CSPs,  only  the 
survey  instruments  for  cholecystectomy  and  elective  total  hip  replacement  went  forward  to  full 
implementation. 

Timing  of  Survey  Instruments 

Initially,  patients  were  to  be  interviewed  on  two  separate  occasions;  six  weeks  and  six  months 
following  discharge.  The  survey  instruments  were  developed  to  cover  the  distinct  period  before 
and  for  six  months  after  discharge.  The  first  questionnaire  would  provide  both  baseline  health 
status  before  hospitalization  or  surgery  and  health  status  at  six  weeks  following  discharge  from  the 
hospital.  The  second  questionnaire  would  cover  health  status  at  six  months  following  discharge 
from  the  hospital.  The  goal  was  to  gather  sufficient  information  at  baseline  to  separate  out  the 
effects  of  patient  characteristics,  and  post  acute  care  services  from  the  effects  of  the  hospitalization 
and/or  procedure  itself  on  the  post  hospital  outcomes. 

During  the  design  phase  two  factors  resulted  in  a  change  of  the  duration  of  patient  follow-up 
and  the  frequency  of  contact  following  hospital  discharge.  First,  the  intake  and  follow-up  survey 
instruments  were  tested  on  several  patients  (approximately  ten  patients  per  condition)  to  determine 
the  time  necessary  to  complete  them.  The  length  of  the  initially  designed  questionnaires  was  over 
one  hour  for  a  functionally  able  individual.  A  frail  individual  would  take  much  longer  to  complete 
the  questionnaires.  The  second  factor  was  the  recommendation  by  the  clinical  panels  that  follow-up 
for  three  of  the  four  conditions  take  place  for  one  year  following  discharge  instead  of  six  months. 

These  concerns  were  presented  to  the  Advisory  Committee  which  made  two  recommendations 
to  reduce  the  amount  of  time  that  respondents  spent  on  the  telephone.  They  suggested  that 
questionnaires  be  shortened  by  eliminating  many  questions  and  that  the  six  week  survey  be  divided 
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into  two  survey  instruments  to  be  administered  at  two  separate  time  periods  instead  of  one.  By 
dividing  the  questionnaire,  the  amount  of  time  the  respondent  would  be  on  the  telephone  during  the 
initial  interview  was  shortened.  The  two  week  interview  was  deemed  necessary  to  collect 
information  about  patient  status  prior  to  hospitalization  and  symptoms  on  admission,  information 
that  was  not  expected  to  be  regularly  recorded  in  hospital  medical  records.  Dividing  the  initial 
survey  had  the  added  benefit  of  addressing  the  Advisory  Committee's  concern  about  the 
recommendation  of  the  CSPs  for  the  need  for  one  year  of  follow-up.  The  Advisory  Committee  felt 
that  allowing  almost  one  full  year  between  intake  and  follow-up  without  an  intervening  contact  was 
too  long.  Thus  they  proposed  adding  an  additional  survey  of  patients  at  the  mid-point  of  the  year 
long  follow-up.  The  mid-point  survey  would  include  the  survey  questions  from  the  initial 
interview  that  dealt  with  current  status.  Because  of  uncertainty  about  the  appropriate  time  for  the 
mid-point  interview,  the  sample  for  each  condition  was  divided  into  thirds;  one  third  each  was 
interviewed  at  four,  five,  and  six  months  following  discharge.  The  four  to  six  month  and  one  year 
survey  instruments  were  designed  to  gather  outcome  data  at  two  points  following  hospitalization. 
The  three  survey  points  became  two  weeks,  four  to  six  months,  and  one  year  post  hospitalization. 

Despite  the  inclusion  of  an  extra  data  point,  there  was  still  concern  about  the  length  of  the 
questionnaires.  To  reduce  time  further,  additional  questions  were  deleted  and  significant  skip 
patterns  were  developed.  The  criteria  used  in  deleting  questions  included  the  complexity  of  the 
questions,  the  difficulty  of  asking  particular  questions  over  the  telephone,  and  the  importance  of 
the  information  to  the  overall  analysis.  For  example,  questions  on  the  use  of  specific  medications 
were  dropped  since  it  was  felt  that  respondents  might  not  be  familiar  with  medication  names  or 
their  uses.  The  cognitive  questions  were  dropped  entirely,  as  they  are  too  difficult  to  assess. 
Questions  such  as  "how  did  you  climb  stairs"  and  "did  hip  pain  limit  your  sexual  activity,"  while 
important  to  the  clinicians,  were  determined  to  be  less  important  or  too  sensitive  for  purposes  of 
PHOS.  In  several  cases,  similar  questions  were  combined,  thus  shortening  total  length  while 
retaining  general  content.  For  example,  the  original  draft  of  the  survey  instrument  included  one 
question  about  bladder  control  and  one  question  about  use  of  a  catheter.  These  two  questions  were 

8 


combined  in  revisions  of  the  survey  instrument  by  rewording  the  response  categories.  Appendix  F 
Hsts  the  specific  questions  found  in  the  survey  instruments  for  cholecystectomy  and  elective  total 
hip  replacement  (the  two  final  conditions  selected  for  study).  The  table  also  outlines  the  evolution 
of  the  survey  instruments  in  light  of  the  considerations  mentioned  above.  Estimated  time  needed  to 
complete  the  surveys  in  their  final  form  was:  for  cholecystectomy — 30  minutes  baseline,  35 
minutes  for  each  of  the  two  follow-up  interviews;  for  total  hip  replacement — 40  minutes  baseline, 
45  minutes  for  each  of  the  two  follow-up  interviews. 

Method  of  Survey  Data  Collection 

Telephone  vs.  in-person  or  mail 

The  surveys  were  written  for  use  over  the  telephone.  Cost  was  the  primary  reason  for 
choosing  telephone  interviews  over  in-person  interviews.  Mail  questionnaires  were  not  considered 
for  this  study  because  the  response  rate  would  be  lower.  The  significant  number  of  lost  responses 
could  introduce  a  substantial  selection  bias  into  the  results.  In  addition,  a  written  questionnaire 
would  be  difficult  for  frail  respondents  to  complete,  thereby  increasing  the  likelihood  of  refusals  or 
proy-es  completing  the  questionnaire.  Realizing  the  inherent  problems  in  selective  sample  loss  with 
telephone  interviews,  in-person  contacts  were  to  be  made  with  those  individuals  where  refusals 
were  likely  (i.e.,  individuals  in  institutions  who  do  not  have  access  to  a  telephone  and  privacy  and 
patients  or  proxies  who  refuse  the  telephone  interview  or  cannot  be  reached  by  telephone). 

CATI  vs.  paper  and  pencil 

The  questionnaires  as  they  were  developed  were  suitable  for  either  a  traditional  telephone 
survey  or  a  computer  assisted  telephone  interview  (CATI)  method.  The  traditional  method  of 
interviewers  following  a  paper  survey  and  hand  writing  responses  and  comments  has  both  benefits 
and  problems  as  does  the  CATI  method.  The  traditional  method  relies  heavily  on  interviewer 
training  in  order  to  implement  skip  patterns  in  the  survey.  If  the  survey  skip  patterns  are 
complicated,  the  interviewer  could  have  difficulty  in  maintaining  the  flow  of  the  interview  as  well 
as  in  obtaining  accurate  information.  The  CATI  program  is  specifically  designed  to  help 
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interviewers  in  navigating  skip  patterns.  The  skip  patterns  developed  in  the  PHOS  survey 
instruments  were  specifically  designed  so  that  they  were  easily  navigated  by  either  survey  method. 
CATI  provides  immediate  data  entry.  The  CATI  also  verifies  responses  as  they  are  being  entered, 
to  insure  that  they  fall  within  permissible  ranges.  However,  the  CATI  cannot  verify  if  the  response 
is  correct.  Interviewers  are  trained  in  how  to  conduct  the  interview,  focusing  on  the  flow  of  the 
conversation,  how  to  deal  with  difficult  situations  in  which  the  individual  does  not  want  to  respond 
or  is  having  difficulty  responding,  etc.  The  CATI  can  interrupt  the  conversational  flow  developed 
by  interviewers  probing  for  more  information  by  moving  too  quickly.  The  manual  telephone 
interview  method  was  used  due  to  the  increased  expense  and  development  time  required  by  the 
CATI  method. 

Medical  Record  Data 

In  addition  to  the  information  obtained  directly  from  patients  through  the  questionnaires, 
information  was  collected  from  hospital  medical  records  and  Medicare  utilization  data  files.  Hospital 
medical  records  were  abstracted  to  obtain  information  on  severity  of  the  condition  and  use  of 
hospital  services.  Specific  information  collected  included'  1)  preoperative  factors  including 
sociodemographic  characteristics,  comorbidity,  risk  factors,  and  mental  health;  2)  intra-operative 
factors  such  as  type  of  procedure;  and  3)  postoperative  factors  including  care  given  during  the 
hospital  stay,  adverse  events  occurring  during  the  hospital  stay,  and  discharge  status  and 
destination.  This  study  did  not  include  information  from  physician  medical  records,  placing  a  heavy 
reliance  regarding  health  status  prior  to  admission  on  information  gathered  from  the  patient,  and 
from  hospital  admission  notes  and  utilization  of  services  prior  to  hospitalization  (Medicare  data). 

The  CSP  members  developed  lists  of  information  that  should  be  collected  from  the  hospital 
medical  record.  These  lists  were  very  inclusive  (see  Appendix  G).  A  central  issue  in  collecting  data 
from  the  medical  record  was  identifying  the  best  approach  to  be  used  for  the  chart  abstraction.  The 
first  option  was  to  develop  a  method  specifically  for  use  by  PHOS  which  would  permit  collection 
of  any  and  all  relevant  information  (given  that  it  can  be  found  in  the  medical  record).  The  specific 
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methodology  would  not,  however,  be  readily  accessible  to  other  research  efforts  in  the  future.  A 
second  option  was  initially  explored;  to  use  the  relatively  new  UCDS  chart  abstraction 
methodology  in  use  by  some  PROs.  The  advantage  of  using  UCDS  was  that  the  method  used  in 
PHOS  would  possibly  be  a  dominant  method  of  chart  abstracting  in  the  future.  A  discussion  was 
held  with  staff  from  the  Health  Services  Quality  Bureau,  who  indicated  their  willingness  to  make 
modifications  in  the  UCDS  for  use  in  this  study,  expanding  it  to  include  additional  items  needed  by 
PHOS.  While  in  the  process  of  making  those  modifications,  the  intention  by  the  federal 
government  to  support  the  UCDS  system  changed.  Realizing  the  UCDS  would  not  be  the 
dominant  system  used  in  the  future,  it  was  decided  to  develop  a  unique  abstraction  tool  that 
included  all  of  the  items  originally  deemed  important  by  the  CSPs. 

Medicare  Data 

In  addition  to  information  collected  from  surveys  and  medical  records,  Medicare  data  can 
provided  information  on  utilization  and  cost  of  care.  It  can  also  provide  information  on 
sociodemographic  characteristics;  comorbidity  deduced  from  prior  utilization  and  recorded 
diagnoses;  post  discharge  complications  can  be  calculated.  Medicare  administrative  data  ncludes  a 
description  of  services  utilized  and  costs  incurred  by  each  type  of  service,  inpatient,  physician, 
outpatient,  home  health  care,  hospice,  and  durable  medical  equipment.  The  theory  underlying  this 
project  was  that  the  Medicare  administrative  data  could  be  very  useful  in  tracking  patient  outcomes  if 
it  were  coupled  with  clinical  information  that  delineated  risk  factors.  For  example,  a  question  of 
great  salience  is  whether  patients  undergoing  elective  hip  procedures  get  longer  life  from  different 
types  of  prostheses.  By  identifying  the  type  of  procedure  performed  and  the  type  of  prosthesis  used 
(and  being  able  to  adjust  for  patient  characteristics)  it  is  possible  to  track  in  the  Medicare  data  base  to 
compare  the  rates  for  reoperations  on  the  same  hip  in  subsequent  years.  Likewise,  the  Medicare  data 
base  can  be  used  to  track  the  relative  effectiveness  of  laparoscopic  and  open  cholecystectomies  by 
looking  for  evidence  of  subsequent  gall  bladder  problems  of  sequelae  of  the  surgery. 
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Sampling  Frame  and  Methodology — Initial  Plan 

Selecting  Hospitals/Communities 

The  overall  goals  of  the  sampling  design  were  to:  1)  identify  a  nationally  representative 
sample,  2)  insure  an  adequate  sample  size  to  determine  predictors  of  outcomes,  3)  enable  the 
identification  of  the  sample  within  a  six  month  time  frame,  and  4)  include  a  sample  of  urban  and 
rural  hospitals  for  cholecystectomies.  During  the  design  phase  of  the  project  a  sampling  plan  was 
developed  with  the  goal  of  using  a  selected  number  of  hospitals  to  identify  a  nationally 
representative  sample  of  patients  from  a  limited  number  of  geographic  locations.  In  order  to 
achieve  a  nationally  representative  sample  a  two  stage  clustering  technique  was  suggested.  The  first 
cluster  being  Metropolitan  Statistical  Areas  (MSAs)  and  the  second  cluster  being  hospitals  within 
MSAs.  The  original  proposal  suggested  selecting  ten  MSAs  geographically  disbursed  across  the 
country.  The  number  of  hospitals  to  be  recruited  in  each  location  was  calculated  based  upon  the 
total  number  of  cases  to  be  collected  during  the  six  month  data  collection  period  (an  estimated  ten 
hospitals  per  location  or  MSA  to  yield  the  intended  sample  size  of  11 00  cases  per  condition). 

Upon  further  review  of  the  historical  number  of  discharges  by  hospital  for  cholecystectomy 
and  elective  total  hip  replacement  it  was  determined  that  it  would  be  difficult  to  obtain  the  intended 
sample  size  within  the  allotted  six  month  time  frame  by  utilizing  ten  hospitals  in  each  of  ten  MSAs. 
Most  MSAs  did  not  have  ten  hospitals  discharging  enough  patients  in  either  of  the  two  conditions 
being  studied  to  generate  the  necessary  sample  size.  The  plan  was  modified  to  include  more  MSAs 
so  that  those  hospitals  conducting  a  relatively  large  number  of  procedures  would  be  included  while 
eliminating  the  facilities  in  each  MSA  with  very  low  numbers  of  procedures.  The  total  number  of 
participating  hospitals  and  the  number  of  cases  needed  remain  unchanged.  Six  MSAs  were  to  be 
selected  from  each  of  four  census  regions  for  a  total  of  24  MSAs.  Within  each  MSA  approximately 
four  to  six  hospitals  would  be  recruited  from  which  to  collect  the  intended  patient  sample,  for  a 
total  of  approximately  100  hospitals.  Based  upon  prior  experience,  it  was  estimated  that  contacts 
would  need  to  be  made  with  120-150  hospitals  in  order  to  reach  the  goal  of  100  hospitals.  To 
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control  for  hospitals  that  did  an  unusually  large  number  of  procedures  the  number  of  patients 
coming  from  any  single  hospital  was  capped  at  30  cases  per  condition. 

The  sample  was  also  expanded  to  include  rural  hospitals.  In  addition  to  the  1 100 
cholecystectomy  cases  sampled  from  hospitals  within  MSAs,  a  sample  of  400  individuals 
discharged  from  rural  hospitals  for  cholecystectomies  was  added.  The  sample  was  expanded  to 
include  rural  hospitals  for  several  reasons.  Inclusion  of  a  sample  of  rural  hospitals  was  specifically 
requested  by  AHCPR  staff  because  rural  hospitals  represent  a  different  mode  of  practice.  It  was 
important  in  the  study  of  cholecystectomy  that  the  respective  outcomes  of  laparoscopic  surgery  be 
compared  to  traditional  cholecystectomies.  The  diffusion  of  the  technology  is  often  variable  (urban 
fast/rural  slow)  therefore  it  was  anticipated  that  most  urban  hospitals  would  be  performing 
laparoscopic  surgery  when  the  data  collection  phase  of  the  study  occurred.  It  was  likely  that  the 
new  procedure  would  not  be  in  use  as  much  or  as  rapidly  in  rural  hospitals.  By  including  rural 
hospitals  in  the  sample  the  differences  between  urban  and  rural  hospitals  in  introducing  new 
technologies  successfully  could  possibly  be  measured.  Based  upon  the  average  number  of  cases 
per  rural  hospital  per  month  (i.e.  ten  cases  per  month  over  six  months,  or  less  than  two  cases  per 
month)  it  was  estimated  that  40  rural  hospitals  would  be  needed.  Approximately  two  hospitals 
located  just  outside  each  MSA  (within  100  miles  of  the  MSA)  would  be  identified  for  use  in  the 
primary  study. 

Once  recruited,  hospitals  would  identify  eligible  patients  by  condition.  Specifically,  hospital 
staff  were  to  monitor  daily  the  admissions  of  Medicare  beneficiaries  65  and  older  for  appropriate 
admitting  diagnoses  and  discharges  to  the  hospital  which  fell  within  specific  ICD  procedure  codes. 
Names  and  addresses,  etc.  of  all  patients  selected  were  to  be  forwarded  to  the  University  of 
Minnesota  data  collection  center  immediately  upon  discharge. 
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Implementation  Phase 
Pretest 

Following  creation  of  the  survey  instruments  and  sampling  frame,  a  pretest  was  conducted. 
The  purpose  of  the  pretest  was  to  test  of  the  logistics  of  patient  identification  and  the  administration 
of  survey  instruments,  as  well  as  data  collection,  editing,  and  coding  procedures.  All  sampling  and 
survey  procedures  described  earlier  were  tested  and  survey  item  quality  and  response  rates  were 
assessed.  Ease  of  administering  the  questionnaires,  clarity  of  questions,  and  specific  terminology 
was  checked.  Items  that  were  problematic  in  the  survey  instruments  were  identified.  The  length  of 
the  questionnaires  was  reviewed  based  upon  the  ability  of  patients  to  complete  the  survey, 
specifically  looking  at  whether  or  not  patients  recovering  from  surgery  are  able  to  talk  on  the 
telephone  for  30  to  45  minutes.  The  accuracy  of  the  Medicare  Identification  Number  obtained  from 
hospitals  was  verified  by  calling  up  the  records  for  the  sample  to  see  if  there  was  a  Medicare 
hospitalization  for  the  corresponding  date.  The  reliability  of  information  given  by  proxies  was 
assessed  by  comparing  responses  from  a  sample  of  patients  to  those  obtained  from  the  designated 
proxy  for  those  individuals.  As  a  result  of  the  pretest,  specific  guidelines  for  interviewers  were 
developed  regarding  how  to  determine  when  contact  with  a  proxy  is  necessary  because  a  patient  is 
too  mentally  confused  or  cognitively  impaired  to  be  a  reliable  respondent.  Procedures  to  identify 
and  track  patients  following  hospital  discharge  and  between  interviews  were  refined,  and  the  ability 
to  obtain  information  from  rural  hospitals  and  patients  was  verified.  In  the  interest  of  time,  only  the 
baseline  and  midpoint  interviews  were  conducted  and  the  midpoint  interview  was  conducted  at 
three  months  following  discharge. 

A  major  issue  in  the  sampling  methodology  was  the  ability  to  recruit  the  required  number  of 
hospitals  both  in  urban  and  rural  locations.  During  the  pretest,  Milwaukee,  Wisconsin,  and  Little 
Rock,  Arkansas,  were  selected  as  sample  MSAs  because  of  their  geographical  location  and  other 
assumed  characteristics  based  upon  previous  work  in  those  areas.  In  general,  the  south  is 
traditionally  a  difficult  area  to  recruit  hospitals  as  compared  to  the  upper  midwest.  Familiarity  with 
the  University  and  relatively  close  proximity  to  test  in-person  interviews  were  also  factors. 
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The  pretest  began  by  using  a  subset  of  the  original  design;  hospitals  were  recruited  in  the  two 
MS  As  and  the  surrounding  rural  areas.  Support  for  the  project  was  solicited  from  the  state  medical 
associations  and  the  hospital  trade  associations.  Direct  appeals  to  the  hospitals  were  made  through 
letters  and  telephone  calls.  Of  63  potential  hospitals,  27  hospitals  (43%)  were  successfully 
recruited.  It  was  especially  difficult  to  recruit  rural  hospitals  (of  the  37  potential  hospitals  in  the  two 
rural  areas  1 1  (30%)  were  successfully  recruited.  Reasons  given  for  not  participating  included  lack 
of  time,  lack  of  available  staff,  too  much  research  activity  already,  no  interest,  lack  of  trust  of  the 
government  and  academia.  Appendix  H  gives  more  detailed  information  about  characteristics  of  the 
participating  and  non-participating  hospitals  in  those  two  states. 

Patients  were  selected  by  condition  from  the  participating  hospitals.  Specifically,  hospital  staff 
were  asked  to  monitor  daily  the  admissions  of  Medicare  beneficiaries  65  and  older  for  appropriate 
admitting  diagnoses  and  discharges  to  the  hospital  which  fell  within  specific  ICD  procedure  codes. 
Names,  addresses,  etc.  of  all  patients  identified  were  forwarded  to  the  University  of  Minnesota 
data  collection  center  immediately  upon  discharge.  Typical  hospital  personnel  involved  in 
identifying  patients  included  medical  records  staff,  nursing  supervisors,  and  admission  personnel. 

In  addition  to  difficulties  in  recruiting  hospitals,  it  was  apparent  early  in  the  project  that  the 
participating  hospitals  were  under  reporting  cases.  The  reason  for  under  reporting  of  cases  was 
unknown,  although  it  was  thought  to  reflect  lack  of  staff  and  lack  of  interest  among  the  medical 
records  personnel  assigned  to  the  project.  To  determine  the  extent  of  under  reporting  during  the 
study  period,  cases  of  hip  replacements  and  cholecystectomies  for  the  participating  providers  were 
obtained  from  the  National  Claims  History  (NCH)  file.  The  NCH  file  includes  utilization  records 
that  are  available  through  the  Medicare  data  system  shortly  after  the  bills  are  paid  by  Medicare. 
Appendix  I  compares  the  patients  identified  by  hospitals  with  the  patient  claims  submitted  to  HCFA 
and  found  in  the  NCH  file. 
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Sampling  Frame  and  Methodology — Revised 

As  a  result  of  problems  recruiting  hospitals  and  identifying  patients  through  hospitals,  a  new 
method  of  patient  identification  was  developed  during  the  pretest.  This  method  used  the  Medicare 
fiscal  intermediaries  (FI)  to  identify  patients  who  had  undergone  one  of  the  two  procedures  at  the 
time  the  bill  was  originally  submitted  by  the  hospital  to  the  FI.  Using  FIs  to  identify  cases  was 
tested  in  Wisconsin,  Arkansas,  North  Carolina,  Nevada,  and  Illinois. 

Information  needed  to  allow  patient  identification  and  medical  record  abstraction  was  collected 
direcdy  from  the  patient's  initial  bill  received  from  the  FIs.  Information  found  on  the  patient  bill 
included:  full  name,  address,  procedure  codes,  diagnosis  codes.  Medicare  status  and  identification 
number,  sex,  date  of  birth,  date  of  admission,  date  of  discharge,  insurance  status,  medical  record 
number,  and  hospital  provider  number  (sample  form  included  in  Appendix  J).  National  CD  ROM 
telephone  directories  and  directory  assistance  were  utilized  to  obtain  the  patient's  telephone  number. 

This  method  was  found  to  be  much  more  reliable  and  efficient:  1)  fiscal  intermediaries  were 
recruited  more  easily  than  hospitals;  2)  the  quality  of  the  data,  especially  the  Medicare  patient 
identification  number  (HIC  number),  was  better  when  reported  by  the  intermediaries;  3)  the 
volume  of  cases  generated  by  FIs  was  greater;  and  4)  the  printing  of  paper  copies  of  bills  and 
sending  the  information  to  the  University  of  Minnesota  required  a  minimal  amount  of  effort  on  the 
part  of  FIs.  Appendix  K  compares  the  accuracy  of  HICs  submitted  by  the  hospitals  and  the  FIs 
with  the  information  provided  in  the  HCFA  Enrollment  Database.  No  other  outcomes  research 
project  has  used  this  method  of  patient  identification.  Appendix  L  indicates  the  number  of  hospitals 
represented  in  the  pretest  when  identifying  patients  by  FI  claim  and  their  characteristics. 

Sampling  based  upon  claims  data  from  FIs  raised  the  issue  of  timeliness.  Originally,  baseline 
data  was  to  be  collected  within  two  weeks  of  discharge.  The  baseline  interview  was  specifically 
designed  to  gather  information  on  the  respondent's  status  prior  to  hospitalization  for  comparison  to 
later  post-hospitalization  status.  Therefore,  it  needed  to  be  completed  as  early  as  possible  following 
discharge  to  accurately  capture  information  about  pre-hospitalization  status.  During  the  pretest,  the 
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mean  length  of  time  between  date  of  discharge  and  the  date  information  was  received  at  the 
University  of  Minnesota  was  38  days  and  close  to  85  percent  of  the  FI  sample  was  received  within 
60  days  from  discharge  (Appendix  M). 

Because  the  baseline  interviews  includes  questions  about  pre-hospitalization  status,  it  was 
important  to  test  how  recall  was  altered  by  a  delay  in  identifying  patients  following  discharge.  This 
comparison  was  important  because  using  the  FIs  meant  some  delay  in  identifying  patients  if 
hospitals  did  not  submit  bills  promptly.  Hence,  some  interviews  were  done  later  than  the  originally 
planned  two-week  window.  To  determine  the  extent  of  recall  differences,  during  the  pretest  a 
subset  of  respondents  (approximately  20  per  condition)  were  administered  the  baseline  survey  as 
the  initial  interview  as  well  as  three  months  following  discharge.  Appendix  N  displays  the 
comparisons  of  the  responses  at  two  weeks  after  discharge  and  at  three  months  following 
discharge.  The  information  is  arranged  to  show  the  average  size  of  the  differences  between  the 
responses  at  the  two  time  periods  as  a  function  of  the  size  of  the  range  of  possible  responses.  In 
general,  the  differences  represent  a  very  small  fraction  of  the  scale  range  (on  the  order  of  5-10 
percent),  suggesting  that  the  two  approaches  produced  comparable  results. 

Patient  Selection 

The  respondent  universe  included  all  elderly  non-HMO  enrolled  Medicare  beneficiaries,  age 
65  and  over,  discharged  following  one  of  two  procedures;  cholecystectomy  and  elective  total  hip 
replacement  from  hospitals  served  by  selected  FIs.  Printed  copies  of  claims  were  sent  from  the  FI 
to  the  University  of  Minnesota  approximately  twice  a  week  via  ovemight  mail.  Staff  at  the 
University  of  Minnesota  screened  each  claim  for  a  valid  Medicare  claim  number  (HIC),  age  of 
patient,  admitting  diagnosis,  HMO  status,  and  length  of  time  from  hospital  discharge  to  receipt  at 
University  of  Minnesota.  Individual  patient  HICs,  first  name,  last  name,  and  date  of  birth  provided 
by  the  FIs  were  validated  against  HCFA's  Medicare  Enrollment  Database  (EDB)  The  EDB  is  a  file 
which  contains  identifying  information  on  all  currently  enrolled  beneficiaries. 
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HMO  members  were  omitted  from  the  sample  because  data  on  the  hospitaHzation  of  HMO 
members  received  by  HCFA  are  incomplete.  Obtaining  comparable  data  from  the  managed  care  plans 
would  be  costly,  difficult  to  obtain,  and  in  some  cases,  not  available  since  they  are  not  required  to 
report  cost  data.  It  is  estimated  that  at  the  time  this  study  began  only  7  percent  of  Medicare 
beneficiaries  are  enrolled  in  HMOs  (of  which  5  percent  were  in  risk-based  HMOs).  Therefore  it  is 
difficult  to  capture  enough  HMO  enrollees  from  a  generalized  nationally  representative  sample.  HMO 
enrollment  is  not  evenly  distributed  across  the  country.  For  example.  Medicare  HMOs  are  very  active 
in  southern  California  and  Minnesota  but  much  less  so  in  New  England  and  the  South.  Medicare 
HMO  enrollees  are  also  more  prevalent  in  urban  areas  and  rare  in  rural  communities.  With  such  an 
uneven  distribution  of  enrollees,  it  would  be  very  difficult  to  capture  enough  HMO  cases  from  every 
area  that  had  possible  access  to  an  HMO  to  treat  HMO  membership  as  a  full  explanatory  variable  in 
the  analysis.  Oversampling  from  areas  where  there  is  high  market  penetration  among  Medicare 
beneficiaries  would  yield  a  sufficient  sample  but  would  lose  generalizability.  The  sample  would  be 
useful  to  contrast  HDvIO  and  non-HMO  patients,  but  would  require  additional  corrections  for  city 
effects  (e.g.,  the  availability  of  nursing  home  beds  or  the  ratio  of  surgeons  to  population  or  style  of 
practice  or  the  very  fact  that  a  lot  of  the  Medicare  population  is  enrolled  in  HMOs).  If  a  sample 
population  could  be  identified,  only  survey  and  medical  record  data  would  be  available  without  links 
to  utilization  or  cost  data.  Critics  could  assert  that  omitting  HMO  members  slightly  limits 
generalizability  of  the  PHOS  to  all  Medicare  beneficiaries.  However,  in  a  study  of  the  outcomes 
attributable  to  the  post-hospital  care  received  by  a  sample  of  Medicare  patients,  the  University  of 
Minnesota  Post  Acute  Care  Study  found  that  membership  in  an  HMO  had  no  consistent  effect  on 
either  type  of  post-hospital  care  received  or  the  outcomes  of  that  care  (Kane,  1994). 

The  specific  ICD  procedure  codes  used  for  the  two  conditions  were  as  follows: 
Cholecystectomy:  51.22  total  cholecystectomy  and 

51.23  laparoscopic  cholecystectomy 
Elective  Total  Hip  Replacement:  8 1 .5 1  total  hip  replacement 
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The  study  of  cholecystectomy  was  Hmited  to  cholecystectomies  performed  as  the  primary 
procedure.  Patients  who  had  a  secondary  or  incidental  cholecystectomy  were  removed  from  the 
study.  The  study  of  elective  total  hip  replacements  was  limited  to  only  those  individuals  with 
diagnoses  of  osteoarthritis.  As  a  result  patients  with  hip  fractures  (ICD-9-CM  codes  835.0-835.13 
and  820.0-820.9),  rheumatoid  or  other  inflammatory  arthritis  (714.0),  traumatic  arthritis  (716.10- 
716.19),  congenital  hip  dysplasia  (754.30,755.63),  Legg-Calve-Perthes  disease  (732.1),  or 
Paget's  Disease  (73 1 .0)  were  excluded  from  the  sample.  Revisions  of  prior  hip  replacement 
procedures  (8 1 .53)  or  bilateral  operations  where  the  first  hip  was  operated  on  within  one  month 
were  also  excluded.  The  intent  in  defining  the  samples  in  this  manner  was  to  collect  a  clinically 
homogeneous  sample  that  can  be  readily  understood  for  purposes  of  generalizability.  Five  percent 
of  cholecystectomies  and  three  percent  of  total  hip  replacements  were  removed  from  the  pretest 
sample  for  these  reason. 

The  focus  of  this  study  was  on  patients  who  were  discharged  alive.  However,  to  get  a  full 
description  of  outcomes,  hospital  charts  were  abstracted  on  any  Medicare  patient  who  died  while  in 
the  hospital  or  prior  to  our  contacting  th^m  within  two  months  following  discharge.  During  the 
pretest  less  than  2  percent  of  cases  died  before  the  initial  interview. 

Pretest  Results 

During  the  pretest  claims  data  for  1671  patients  were  received  by  the  University  of  Minnesota. 
Of  that  number  just  over  half  (54%)  were  determined  to  be  eligible  for  the  study.  Screening  the 
claims  for  accuracy  up  front  is  essential  based  upon  this  method  of  identification.  Patients  were 
excluded  based  upon  our  selection  criteria,  age,  and  being  received  outside  of  the  intended 
interview  window.  Of  the  eligible  sample,  96  percent  of  the  patients  were  successfully  located 
(Appendix  O).  From  those  863  patients  located,  survey  interviewers  attempted  to  complete  the 
baseline  and  follow-up  interviews  on  a  smaller  subset,  359  individuals  (195  cholecystectomy 
patients  and  164  elective  total  hip  replacement).  Of  those  359  potential  respondents,  96  percent 
completed  the  baseline  survey,  four  percent  refused  to  participate.  Another  six  percent  of  the 
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sample  did  not  complete  the  follow-up  interview  either  because  they  refused,  were  deceased  or 
their  location  had  changed.  The  overall  response  rate  was  90  percent  (Appendix  P). 

Two  central  issues  addressed  in  the  pretest  were  the  ability  to  locate  patients  based  upon 
claims  data  (which  did  not  include  telephone  numbers)  and  the  ability  to  track  patients  over  time. 
From  the  total  FI  eligible  sample  of  895  for  both  conditions,  863  patients  or  proxies  (96  percent) 
were  located,  (Appendix  O).  Of  the  cases  for  whom  a  telephone  number  was  obtained,  68  percent 
of  the  sample  was  located  by  using  telephone  directory  assistance,  3 1  percent  by  hospital  inquiries, 
and  1  percent  by  other  sources  such  as  relatives.  Once  located,  less  that  7%  of  the  sample  was  lost 
to  follow-up  within  the  three  month  time  frame  used  in  the  pretest.  Of  that  number,  2%  were  due  to 
death.  Expectations  for  the  full  study  were  set  based  upon  this  information  (Appendix  P). 

During  the  pretest  the  accuracy  and  reliability  of  proxy  responses  was  tested  against  patient 
interviews.  Both  patients  and  proxies  were  contacted  at  baseline.  Responses  were  compared.  The 
level  of  agreement  was  from  30  percent  to  100  percent  and  virtually  all  were  significant  (Appendix 
Q).  Based  upon  these  results  the  use  of  in-person  interviews  was  eliminated.  The  decision  was 
made  to  use  proxies  where  the  individual  was  unavailable  or  unable  to  answer  the  survey 
questions.  Specific  criteria  were  established  to  assist  interviewers  decide  the  need  for  a  proxy.  The 
goal  was  to  avoid  proxy  interviews  whenever  possible.  During  the  pretest  89  percent  of  the 
surveys  were  completed  by  telephone,  2  percent  by  in-person  interviewers,  and  9  percent  by  proxy 
(Appendix  R). 

Full  Implementation 

Sampling  Frame  and  Methodology 

The  full  study  began  in  September  of  1994.  Given  the  success  during  the  pretest  in  using 
fiscal  intermediaries,  the  full  study  employed  the  same  method  to  identify  patients.  Cases  were 
identified  through  12  fiscal  intermediaries  (FIs)  (Appendix  S)  who  historically  processed  the 
greatest  number  of  bills  for  cholecystectomy  and  hip  replacement  (originally  thirteen  FIs  were 
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selected  but  one  declined  to  participate  due  to  resource  limitations).  The  rationale  for  this  sample 
was  to  obtain  a  nationally  representative  sample  that  would  also  generate  enough  cases.  These  FIs 
covered  many  states,  but  we  restricted  the  data  used  for  this  study  to  those  states  where  the  FI 
serviced  over  half  the  Medicare  population.  These  states  are  shown  in  Appendix  S.  In  one  case  an 
FI  covered  more  than  half  of  three  states  and  we  elected  to  draw  patients  from  only  one  of  these, 
Mississippi.  The  FIs  began  sending  claims  data  in  late  October  1994.  Sample  intake  took 
approximately  eight  months  to  complete. 

To  determine  predictors  of  outcomes  an  adequate  sample  size  was  needed.  For  each  condition 
PHOS  sampled  over  1,100  patients  from  urban/suburban  locations  throughout  the  12  states  served 
by  the  participating  FIs.  An  additional  sample  of  over  400  patients  from  rural  hospitals  was  used 
for  cholecystectomy. 

Claims  were  screened  by  staff  at  the  University  of  Minnesota  based  upon  established 
eligibility  criteria  including  age,  secondary  procedures  and  primary  diagnoses,  length  of  time 
between  discharge  and  receipt  in  Minnesota,  and  bad  information  such  as  incorrect  HIC  number. 
All  potential  cases  were  matched  against  the  HCFA  Enrollment  Database  to  verify  the  HIC  number. 
Once  the  case  was  included  in  the  study  sample,  interviewers  attempted  to  locate  a  correct 
telephone  number  and  contact  the  individual.  Respondents  were  surveyed  within  two  months  of 
discharge  from  the  hospital,  at  4  to  6  months  following  discharge,  and  at  one  year  following 
discharge. 

The  exact  number  of  subjects  to  include  from  each  FI  was  determined  based  upon  historical 
data  regarding  the  annual  number  of  patients  with  each  diagnosis  discharged  from  hospitals  in 
states  served  by  each  FI.  The  number  of  patients  to  be  discharged  in  the  ten  month  period  of  data 
collection  was  estimated  and  the  correct  sampling  fraction  within  each  FI  was  developed  so  as  to 
sample  proportional  to  size  and  to  keep  the  data  collection  period  the  same  for  all  FIs. 

Next  the  cases  were  stratified  by  whether  the  hospital  was  in  an  urban  or  rural  area  (rural 
defined  by  the  definition  used  in  the  HCFA  Provider  of  Services  file.)  This  allowed  for  a  sufficient 
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sample  of  cholecystectomy  cases  obtained  from  rural  hospitals.  The  proportion  of  proprietary, 
non-profit,  local  government-owned,  teaching,  small  and  large  hospitals  included  in  the  sample 
was  expected  to  be  nationally  representative  across  the  sampled  FIs.  Therefore  it  was  not  necessary 
to  sample  separately  from  stratified  groups  representing  these  characteristics  in  order  to  say 
something  about  different  types  of  hospitals  in  the  analysis.  Appendix  T  discusses  the  design  effect 
related  to  the  sampling  plan. 

The  sampling  design  for  the  full  study  originally  focused  on  inpatient  hospitalizations  and 
used  ICD  procedure  codes  for  identification.  Upon  further  examination  of  the  discharge  rates  for 
the  two  conditions  being  studied  it  was  determined  that  a  significant  number  of  laparoscopic 
cholecystectomies  were  being  performed  on  an  outpatient  or  same  day  surgery  basis.  The  sampling 
method  was  modified  in  order  to  capture  a  sufficient  number  of  inpatient  open  cholecystectomies, 
inpatient  laparoscopic  cholecystectomies,  and  outpatient  laparoscopic  cholecystectomies, 
specifically  500  cases  each.  The  number  of  cases  needed  from  each  FI  was  then  distributed 
proportional  to  the  historical  number  of  discharges  by  each  category..  The  number  of  inpatient  open 
cholecystectomies,  inpatient  laparoscopic  cholecystectomies,  and  outpatient  laparoscopi'^- 
cholecystectomies  was  set  to  be  equal  (500  cases  each)  instead  of  proportional  to  historical 
numbers  for  two  reasons:  first  it  was  believed  that  the  historical  information  did  not  adequately 
reflect  the  changing  nature  of  cholecystectomy  surgery  and  therefore  did  not  adequately  reflect 
current  and  future  numbers  of  procedures  performed  and  second  this  method  would  generate 
enough  cases  in  each  group  to  make  subgroup  comparisons  (Appendix  U). 

In  order  to  accomplish  this  change  the  FIs  were  first  instructed  to  include  outpatient  claims  in 
the  data  being  sent  to  the  University  of  Minnesota.  Second,  the  tracking  system  at  the  University  of 
Minnesota  used  to  screen  eligible  claims  was  modified  to  include  type  of  bill  (bill  type  code 
structure  in  Appendix  V)  and  HCPC  codes  used  for  paying  outpatient  claims.  The  eligible  HCPC 
codes  included:  47600^7620  for  open  cholecystectomy  and  56340-56342  for  laparoscopic 
cholecystectomy. 
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During  the  full  implementation,  University  of  Minnesota  surveyors  attempted  to  locate  and 
contact  the  patient  within  ten  days  of  receiving  information  from  the  FI.  Claims  falling  outside  of 
the  two  months  post-discharge  window  were  omitted  from  the  sample.  During  the  full 
implementation  close  to  90  percent  of  the  FI  sample  was  received  at  the  University  of  Minnesota 
within  60  days  from  discharge  and  approximately  one-half  was  received  within  30  days  (Appendix 
W).  A  total  of  526  cases,  7.7%  of  the  total  intake  sample,  were  excluded  from  the  sample  because 
of  being  outside  the  two  month  window. 

Medical  Record  Abstraction 

With  the  endorsement  of  HSQB,  Peer  Review  Organizations  (PRO)  in  each  of  the  states/areas 
served  by  the  participating  FIs  were  contacted  to  participate  in  the  study.  Conducted  as  a 
cooperative  project  under  the  PRO  contract  with  the  Health  Care  Financing  Administration  and  as 
part  of  their  regular  PRO  review  activities,  PROs  were  asked  to  obtain  copies  of  the  medical 
records  for  all  patients  who  completed  the  baseline  interview  (or  proxies  completed  the  interview) 
or  who  were  deceased  prior  to  baseline.  Medical  records  were  forwarded  on  to  a  central  location 
(the  Colorado  Foundation  for  Medical  Care)  for  abstraction.  Hospital  records  were  abstracted 
using  a  tool  especially  designed  for  this  project  using  guidelines  established  during  the  design 
phase  of  the  project  by  the  clinical  experts  (the  complete  abstraction  tool  is  located  in  the  Data 
Collection  Manual).  The  instrument  and  training  for  abstractors  was  tested.  Revisions  were  made 
until  reliability  for  each  individual  item  was  90  percent  or  greater.  Reliability  of  the  instrument  and 
abstractors  was  checked  periodically  during  the  review  process. 

Results 

Sampling  Frame 

Appendix  X  indicates  the  number  of  completed  surveys  at  each  data  point.  The  initial  eligible 
sample  included  2701  cholecystectomies  and  1640  total  hip  replacements.  The  final  sample  used  to 
gather  other  sources  of  data  including  hospital  medical  record  and  Medicare  data  was  based  upon 


23 


those  individual  completing  the  baseline  interview  and  those  individuals  deceased  at  baseline,  1 569 
cholecystectomies  and  1 133  total  hip  replacements. 

A  ten  percent  loss  in  the  sample  over  the  course  of  the  project  was  anticipated.  In  fact,  the  hip 
replacement  sample  declined  6%  and  the  cholecystectomy  sample  declined  9.8%  from  baseline  to 
one  year.  Of  those  1,423  patients  (947  cholecystectomy  patients  and  the  476  total  hip)  who  did  not 
complete  any  of  the  three  survey  interviews  62  patients  (49  cholecystectomy  and  13  hip)  died  prior 
to  baseline,  26  patients  (17  cholecystectomy  and  9  hip)  died  prior  to  the  midpoint,  and  40  patients 
(27  cholecystectomy  and  13  hip)  died  prior  to  the  one  year  follow-up.  According  to  medical 
records  information,  four  hip  patients  and  eleven  cholecystectomy  patients  died  in  the  hospital.  In 
addition,  respondents  were  labeled  incomplete  if  the  survey  interviewers  were  unable  to  locate  a 
valid  telephone  number,  the  interviewers  were  able  to  locate  the  individual  but  were  unable  to 
complete  the  interview  within  the  specified  time  window,  the  individual  refused,  or  there  was  no 
proxy  available.  Appendix  X  also  indicates  the  proportion  of  cases  failing  to  complete  the  surveys 
at  the  three  data  points  for  the  above  reasons.  Although  not  specifically  tracked  during  the  course 
of  the  project,  the  loss  of  sample  by  FI  was  fairly  consistent  across  the  FIs,  as  indicated  in 
Appendix  Y. 

To  test  the  adequacy  of  the  data  collection  procedures  separate  analyses  were  performed  on 
each  sample.  All  analyses  were  performed  by  using  a  logit  specification  and  regressing  personal 
characteristics  (gender,  length  of  stay  and  age — all  information  obtained  from  the  FI  claims  data) 
along  with  dummy  codes  for  each  of  the  fiscal  intermediaries  from  whom  the  data  was  collected. 
Gender  was  coded  1  for  female,  0  for  male.  Length  of  stays  was  coded  in  days,  and  age  was 
coded  in  years.  First  an  analysis  was  conducted  showing  the  differences  between  those  subjects 
that  were  deemed  eligible  and  were  received  in  a  timely  manner  and  who  completed  the  initial 
baseline  interview  and  those  subjects  that  were  deemed  eligible  and  were  received  in  a  timely 
manner  but  a  baseline  survey  was  not  completed.  Both  the  cholecystectomy  and  total  hip  equations 
were  significant  as  shown  in  Appendix  Z  (cholecystectomy:  Model       184.720,  df=15,  p<.000; 
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Total  Hip  Replacement:  Model  X2=8 1.658,  df=14,  p<.000).  For  both  samples,  females,  those 
patients  with  longer  lengths  of  stay,  and  older  subjects  were  less  likely  to  complete  the  survey. 
Additionally,  for  the  cholecystectomy  sample,  those  who  had  a  laparoscopic  surgery  were  more 
likely  to  complete  the  survey  than  those  receiving  a  traditional  surgery.  There  was  significant 
variation  among  the  FIs  but  it  was  less  pronounced  than  in  the  other  samples  studied.  On  the  other 
hand,  there  is  no  reason  to  suspect  that  the  FI  variable  would  be  significant  in  this  equation. 

Using  claims  data  supplied  by  FIs  was  a  fully  efficient  method  of  identifying  patients.  A 
thorough  screening  of  claims,  however,  was  necessary  to  identify  incorrect  information.  Appendix 
AA  shows  the  number  of  people  excluded  from  the  sample  for  various  reasons:  36.5%  of  the  cases 
had  a  bad  HIC  number  or  other  bad  data;  18.4%  were  too  young;  and  38.2%  of  cases  were 
excluded  because  they  did  not  meet  the  procedure  and  diagnosis  rules.  Overall,  36.3%  of  cases 
received  in  Minnesota  were  excluded  from  the  study,  leaving  the  remaining  eligible  sample  of 
2,701  for  cholecystectomy  and  1,640  for  hip  replacement. 

A  distinction  should  be  made  between  exclusion  from  the  sample  on  the  basis  of  not  meeting 
the  study  criteria  (i.e.,  ineligibility;  and  loss  from  the  study  because  of  notification  after  the  target 
time  window.  Those  in  the  latter  case  met  the  study  criteria  and  hence  are  part  of  the  basic  sample, 
but  they  were  not  included  because  the  notification  of  their  hospitalization  arrived  after  the  time 
window  for  the  baseline  interview.  These  cases  pose  a  potential  bias  to  the  sample  to  the  extent  that 
the  characteristics  that  made  their  notification  late  also  distinguished  them  for  those  with  more 
timely  notification. 

Beneficiaries  who  were  forwarded  to  the  University  of  Minnesota  and  met  all  the  eligibility 
criteria  were  compared  to  those  that  were  determined  to  be  ineligible  using  a  logit  specification 
model.  Both  the  cholecystectomy  and  the  total  hip  replacement  equations  were  significant 
(Cholecystectomy:  Model  X2  =806.610,  df=15,  p<.000;  Total  Hip  Replacement:  Model 
X2=15 1.836,  df=14,  p<.000).  The  results  of  this  analysis  are  shown  in  Appendix  AB  in  columns 
1  and  3  of  the  table  for  cholecystectomy  and  hip  replacement  respectively.  Examination  of  these 


25 


columns  shows  that  those  who  were  inehgible  had  significantly  shorter  lengths  of  stay,  were  older 
than  the  eligible  sample  and,  for  the  cholecystectomy  sample,  were  more  likely  to  have  had  a 
laparoscopic  surgery.  In  addition,  there  was  significant  variation  between  FIs  in  the  number  of 
ineligible  sample  they  forwarded. 

A  second  analysis  looked  at  eligible  sample  members  who  arrived  in  Minnesota  too  late  to  be 
interviewed  (columns  2  and  4  in  Appendix  AB).  Again,  using  a  logit  specification  model  with  the 
dependent  variable  coded  0  for  eligible  subjects  and  1  for  ineligible  subjects,  both  the 
cholecystectomy  and  total  hip  replacement  equations  were  significant  (Cholecystectomy:  Model 
X2=209.062,  df=14,  p<.000;  Total  Hip  Replacement:  Model  X2=71.821,  df=14,  p<.000).  While 
the  overall  results  of  both  the  cholecystectomy  and  total  hip  replacement  samples  were  significant, 
there  are  no  variables  of  interest  that  are  significant.  These  was  significant  variation  between  FIs  in 
their  ability  to  identify  and  forwarded  sample  patients  in  a  timely  manner. 

Of  those  patients  determined  to  be  eligible  for  the  study,  456  or  10.5  percent  were  never 
found.  The  information  provided  by  the  PI  was  incorrect  or  insufficient  to  locate  the  individual. 
The  individuals  that  were  located  were  primarily  found  using  a  CD  ROM  phone  disk  directory.  The 
second  most  often  used  source  of  information  was  directory  assistance  (Appendix  AC).  In  some 
cases  neighbors  provided  forwarding  addresses  or  special  efforts  had  to  be  made  to  track  down  the 
respondents. 

The  use  of  proxies  in  the  full  study  was  somewhat  similar  to  the  experience  in  the  pretest.  In 
1 1  percent  of  the  cholecystectomy  cases  a  proxy  was  used  while  6  percent  were  used  in  total  hip 
replacement  cases  (Appendix  AD).  The  lower  use  of  proxies  for  hip  replacement  cases  reflects  the 
better  health  status  of  patients  undergoing  this  elective  procedure. 

The  resulting  sample  yielded  about  two  cases  per  condition  per  hospital  instead  of  the  ten 
originally  planned,  but  the  goal  was  not  to  examine  cases  in  specific  hospitals.  Appendix  AE 
indicates  the  number  of  hospitals  per  FI.  Appendix  AF  indicates  the  average  number  of  cases  and 
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range  per  hospital  per  FI.  Finally,  Appendix  AG  shows  the  number  of  hospitals  participating  from 
each  FI  by  various  hospital  characteristics. 

Medical  Record  Data 

Hospital  medical  record  data  was  requested  for  all  patients  who  completed  the  baseline 
interview  or  were  deceased  at  baseline.  Of  the  2702  records  requested,  2666  were  received  and 
2602  were  abstracted,  98.7  percent.  Once  a  record  was  received  for  abstraction  it  was  screened 
again  to  see  if  the  patient  met  the  eligibility  criteria.  In  addition  to  reliability  testing  that  was 
completed  during  development  of  the  instrument,  ten  percent  of  all  of  the  records  abstracted  were 
abstracted  by  a  second  reviewer.  Overall  reliability  was  within  3.65  percent  for  total  hip 
replacement  and  under  2  percent  for  cholecystectomy. 

Medicare  Data 

As  with  medical  record  data.  Medicare  utilization  files  were  requested  for  all  respondents 
completing  the  baseline  survey  and  for  those  patients  who  were  deceased  at  baseline.  The  request 
included  the  following  files:  Denominator  (which  indicates  if  the  individual  is  a  member  of  an 
HMO),  MEDPAR  (for  inpatient  utilization  data)  Hospice,  Outpatient,  HHA  (Home  Health 
Agencies),  DME  (Durable  Medical  Equipment),  and  Physician  Part  B  (which  also  includes  clinical 
lab  data).  Data  was  obtained  for  the  period  one  year  prior  to  through  one  year  following  the  date  of 
discharge  for  the  two  procedures  being  studied.  Not  all  patients  have  data  for  a  full  year  following 
the  discharge  date  due  to  the  fact  that  requests  for  Medicare  data  take  some  time  to  process.  The 
request  was  submitted  so  as  to  include  as  many  patient  records  as  possible  and  still  receive  the 
information  before  the  end  of  the  project. 

Records  for  all  but  25  hip  patients  and  14  cholecystectomy  patients  were  located  in  the  files 
sent  from  HCFA.  Even  though  the  HIC  numbers  were  checked  for  accuracy  against  the  Medicare 
Enrollment  Database  before  patients  were  entered  into  the  study,  in  some  cases  it  was  difficult  to 
identify  the  correct  HIC  number  prior  to  the  request  for  data  being  submitted  to  HCFA.  PHOS  files 
were  revised  based  upon  information  obtained  while  locating  the  medical  record  and  rechecked 


27 


against  the  Enrollment  Database.  A  resubmission  requesting  data  on  these  39  patients  with  revised 
HIC  numbers  may  be  successful. 

Inpatient  and  outpatient  files  were  checked  to  find  the  sentinel  event  that  caused  the  individual 
to  be  entered  into  the  PHOS  study.  The  sentinel  event  was  found  in  the  MEDPAR  file  for  all  1 106 
hip  patients,  for  which  there  were  records.  The  sentinel  event  was  found  in  MEDPAR  for  968 
cholecystectomy  patients  and  938  patients  in  the  Outpatient  file.  Forty-nine  patients  had  only 
Physician  Part  B  records.  In  addition,  154  patients  having  MEDPAR  or  Outpatient  records  had  no 
Physician  Part  B  records.  Patients  are  expected  to  have  both  a  facility  (inpatient  or  outpatient 
charge)  as  well  as  a  physician  (surgeon)  charge  for  the  sentinel  event.  Absence  of  one  or  the  other 
may  be  due  to  the  provider  billing  late  or  the  claim  being  denied  and  resubmitted  due  to  a  variety  of 
potential  problems.  Appendix  AH  indicates  that  those  cholecystectomy  patients  with  inconsistent 
data  are  not  significantly  different  from  the  rest  of  the  sample. 

All  patients  were  checked  against  the  denominator  file  to  determine  HMO  status.  Seventy-five 
cholecystectomy  patients  were  in  HMOs  at  some  time  during  the  study  period  and  61  total  hip 
replacement  patients  were  in  HMOs  at  some  time  during  the  same  period.  There  is  the  potential  for 
limited  or  incomplete  utilization  data  on  these  individuals. 

Linking  Data 

Linking  and  verifying  critical  information  across  data  sources  was  made  difficult  by 
inconsistent  and  questionable  or  absent  data.  For  example,  64  cases  that  met  the  eligibility 
requirements  based  upon  claims  data  later  were  determined  ineligible  based  upon  medical  record 
data.  This  data  quality  concern  was  most  evident  in  how  inpatient  and  outpatient  status  was 
determined.  Type  of  bill  indicated  on  the  claim  form  was  used  to  mark  a  claim  as  outpatient  for 
sampling  purposes.  There  were  inconsistencies  found  between  type  of  bill  and  ICD  9  procedure 
code  and  HCPC  code  (open  cholecystectomy  listed  as  an  outpatient  claim).  The  same  problem 
arose  when  comparing  bill  type  with  length  of  stay  (Appendix  AI).  There  is  some  evidence  of 
errors  contained  in  the  FI  data  as  shown  by  examining  the  length  of  stay  by  bill  type.  Ninety 
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eligible  cases  who  were  presumably  treated  as  outpatients  have  length  of  stay  of  two  or  more  days. 
To  further  understand  what  occurred,  additional  information  was  abstracted  for  all  laparoscopic 
patients  to  determine  from  hospital  records  the  outpatient,  inpatient  status.  Abstractors  first 
concluded  whether  the  patient  was  originally  a  hospital  inpatient  admission  based  upon  physician 
documentation  and  the  hospital  face  sheet.  If  the  patient  was  admitted  to  an  outpatient  unit  such  as 
observation,  short  procedure  or  stay,  day  or  ambulatory  surgery,  the  abstractors  looked  at  where 
the  patient  went  directly  after  the  post  anesthesia  care  unit/recovery  room  (hospital  bed, 
observation,  or  discharge)  and  if  the  documentation  indicated  the  outpatient  status  converted  to 
inpatient  status  based  upon  physician  orders.  Appendix  AJ  gives  the  additional  information  by 
length  of  stay.  This  information  suggested  that  while  some  of  the  claims  data  may  be  erroneous, 
the  definition  of  outpatient  status  may  also  not  be  clear.  There  appear  to  be  some  cases  that  are  true 
outpatient  cases  while  others  are  only  probable  or  dubious  at  best.  Appendix  AK  compares  the 
definition  of  outpatient  status  by  the  various  data  sources.  While  there  is  considerable  agreement, 
there  are  still  some  questionable  cases. 

Analysis  of  Results 

Appendices  AL,  AM,  and  AN  offer  a  preliminary  glance  at  the  data  from  the  elective  hip 
replacement  cases.  These  are  still  uncorrected  for  missing  data,  although  the  number  of  missing 
responses  for  any  single  question  is  usually  quite  low  (<0.05%).  No  corrections  for  other  factors 
have  been  done,  nor  have  any  scales  been  developed.  The  sample  consisted  primarily  of 
individuals  between  65  and  79  years  of  age.  They  were  predominantly  white.  A  little  over  half 
were  still  married.  Over  60  percent  of  the  hip  patients  were  female  (Appendix  AL).  The  length  of 
stay  in  the  hospital  for  this  procedure  ranged  from  less  than  one  day  (again  pointing  out  potential 
errors  in  the  FI  data)  to  over  40  days  and  averaged  just  over  six  days  (Appendix  AM). 

There  was  substantial  improvement  in  walking  distance,  with  fewer  people  using  an  assistive 
device.  Various  types  of  movements  were  easier.  More  people  left  the  house.  The  only  area  that 
appears  counter-intuitive  was  the  increased  use  of  an  assistive  device  for  socks  and  shoes.  There 
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was  a  general  reduction  in  pain  of  different  types.  Hip  pain  was  expected  to  be  different  after 
surgery  with  less  specific  hip  pain  and  more  thigh  pain  (Appendix  AN). 

Appendices  AO,  AP,  and  AQ  describe  the  initial  results  for  cholecystectomies.  The  sample 
population  tended  to  be  slightly  younger,  the  majority  being  between  65  and  74.  They  were 
predominantly  white  and  slightly  more,  62.2%,  were  still  married.  Again,  over  60  percent  of  the 
sample  were  female  (Appendix  AO).  The  length  of  stay  in  the  hospital,  as  anticipated,  varied 
greatly  between  open  cholecystectomies  and  laparoscopic  cholecystectomies;  laparoscopic 
cholecystectomies  were  predominantly  grouped  between  less  than  one  and  two  days  (30%  were 
done  as  outpatients)  while  the  open  cholecystectomies  ranged  between  less  than  one  day  to  over  40 
days  with  the  median  length  of  stay  just  over  seven  days  (Appendix  AP). 

The  proportion  of  patients  who  reported  a  pain  similar  to  classic  cholecystitis  pain  dropped 
dramatically  but  the  reports  of  individual  types  of  pain  varied.  Symptoms  like  nausea,  vomiting, 
and  belching  improved  but  flatulence  did  not.  Response  to  greasy  and  fatty  foods  and  dairy  foods 
was  less  severe  but  other  foods  still  brought  on  pain  at  least  as  often.  About  three-quarters  of  the 
patiei.ts  had  resumed  their  usual  activities  by  the  time  of  the  first  follow-up  survey  (the  baseline 
survey  completed  within  two  months  of  discharge).  There  was  some  improvement  in  overall 
ratings  of  health  (Appendix  AQ). 

Lessons  Learned 

The  goal  of  PHOS  was  to  develop  an  approach  to  collect  information  on  Medicare  enrollees 
regarding  important  patient  outcomes  at  relevant  time  periods  after  hospital  discharge  and  to  link 
this  outcomes  data  with  Medicare  utilization  data.  The  methodology  used  to  identify  patients  for 
inclusion  in  the  study  and  to  obtain  different  sources  of  data  was  efficient  and  at  relatively  low 
cost.  The  quality  and  accuracy  of  the  initial  claims  data  may  be  suspect  however  which  caused 
difficulty  in  linking  data  sources  and  leads  to  further  research  questions. 

Specifically,  the  method  of  using  fiscal  intermediaries  to  identify  patients  was  an  efficient 
method  to  identify  a  large  national  sample  within  a  relatively  short  time  frame  and  with  minimal 
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cost.  The  original  study  design  included  utilizing  13  FIs  to  identify  patients.  All  but  one  of  those 
FIs  agreed  to  participate.  Recruitment  was  fairly  easy,  with  relatively  minor  expenses  incurred  by 
each  FI.  With  one  less  FI  than  originally  anticipated,  sample  intake  was  still  completed  within  the 
allotted  time  frame,  allowing  for  a  change  in  eligibility  criteria  midway  through  intake  (addition  of 
using  HCPC  and  Type  of  Bill  to  insure  an  adequate  number  of  outpatient  laparoscopic 
procedures). 

There  is  a  potential  concern  regarding  sample  bias.  It  is  difficult  to  say  the  sampling  method 
yielded  a  truly  nationally  representative  sample  because  it  is  impossible  to  weight  the  sample  back 
to  some  identifiable  base.  Sampling  by  FI  does  not  equate  to  a  proportion  of  Medicare  patients  by 
state  because  FIs  serve  only  part  of  states  or  overlap  into  other  states  and  into  states  served  by 
other  FIs.  Another  concern  is  the  completeness  of  the  sample  drawn  by  the  FIs.  There  is  no  means 
to  test  if  the  patients  identified  by  the  FIs  are  all  of  the  possible  patients.  Based  upon  the 
procedures  set  in  place  and  discussions  with  staff  at  the  various  FIs,  the  sample  should  be  pretty 
complete  in  that  all  potential  cases  were  forwarded  to  the  University  of  Minnesota  for  potential 
inclusion  in  the  study.  The  remaining  concerns  then  are  the  cases  that  simply  were  not  submitted 
for  payment  by  providers  in  a  timely  manner  to  be  included  in  the  study.  There  is  no  way  to 
determine  the  extent  of  these  cases. 

While  the  cost  to  identify  patients  was  fairly  low,  there  was  a  higher  cost  to  screen  the  claims. 
Over  36%  of  the  cases  forwarded  to  the  University  of  Minnesota  were  determined  to  be  ineligible 
for  the  study.  In  addition,  another  8%  were  lost  from  the  sample  due  to  the  late  submission  of  the 
claim.  Although  the  individuals  met  all  eligibility  criteria,  their  claims  were  received  after  the 
established  interview  window  had  passed. 

Before  we  attempt  to  attach  any  meaning  to  these  results,  we  must  remind  ourselves  of  the 
limitations  of  these  analyses.  First,  the  sample  size  for  the  eligibility  analyses  are  large  and  we 
should  depend  upon  the  size  of  the  significant  odds  ratios  rather  than  on  the  fact  that  they  are 
significant.  Second,  we  need  to  be  sensitive  to  the  paucity  of  variables  at  the  patient  level  which  we 
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could  control,  and  more  important,  the  ones  we  could  not  control  such  as  frailty,  residence,  and 
cognitive  ability  all  of  which  most  likely  share  variance  with  our  measured  variables  (i.e.,  gender, 
age,  and  length  of  stay).  Lastly,  we  need  to  remain  cognizant  of  the  obvious  differences  that  exist 
among  fiscal  intermediaries.  This  includes  their  workload,  profitability,  and  the  sufficiency  of  their 
management  information  systems. 

In  general,  we  found  very  large  differences  in  the  ability  of  FIs  to  provide  timely  data  and,  for 
cholecystectomies,  differences  in  their  ability  to  adequately  screen  data  internally  for  even  the 
simplest  variables.  Any  analysis  of  this  dataset  should  control  for  the  effects  of  individual  FIs. 
This  is  especially  true  given  the  volumes  written  on  small  area  variation  and  the  vast  differences  in 
Medicare  reimbursement  across  the  country.  Failure  to  control  for  these  effects  would  likely  lead  to 
severe  bias. 

As  for  the  patient  level  variable  for  which  we  estimated  coefficients,  we  cannot  make  any 
general  conclusions.  For  both  receiving  data  on  time  and  receiving  "good"  data,  we  would 
generally  expect  the  FI  variables  to  capture  most  of  the  variance  (which  they  do).  This  is  because  it 
is  the  IT  who  organized  the  data  for  delivery  to  the  research  team.  It  is  not  clear  why  either 
eligibility  or  timeliness  of  data  delivery  would  be  affected  by  any  of  the  variables  we  measured.  On 
the  other  hand,  it  is  reasonable  to  believe  that  age  is  a  proxy  for  many  other  variables,  especially 
frailty,  as  would  be  length  of  stay.  The  gender  results  are  also  likely  to  be  proxies  for  other 
unmeasured  variables;  this  in  light  of  the  fact  that  women  generally  respond  at  higher  rates  than 
men,  contrary  to  our  finding. 

Once  entered  into  the  sample,  a  large  proportion  of  the  respondents  were  located  and  agreed  to 
participate.  Losses  because  of  no  telephone  number,  and  difficulty  getting  elderly  patients  to 
participate  based  upon  a  'cold'  telephone  call  were  modest.  Between  the  two  conditions,  the 
number  of  respondents  who  were  never  found  due  to  lack  of  a  telephone  number  was  10%  and  the 
number  refusing  to  participate  was  15%. 
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Utilizing  the  Peer  Review  Organizations  to  obtain  copies  of  the  medical  records  was  also  a 
fairly  efficient  and  effective  way  to  accomplish  this  major  task.  Most  hospitals  contacted  accepted 
the  PROs  authority  to  obtain  the  record  as  part  of  their  ongoing  research  activities.  All  but  1.3%  of 
the  records  requested  were  received. 

The  sample  was  stratified  to  include  equal  numbers  of  inpatient  open  cholecystectomies, 
inpatient  laparoscopic  cholecystectomies,  and  outpatient  laparoscopic  cholecystectomies. 
Identifying  inpatient  and  outpatient  status  was  complicated  by  a  lack  of  conjunction  between  FI 
data,  medical  record  data,  and  HCFA  data.  There  is  a  need  for  a  clear  definition  of  ambulatoi7 
surgery  and  a  better  understanding  of  the  application  of  the  definition  versus  billing  procedures 
used  by  providers.  There  is  some  cause  for  concern  in  trying  to  compare  the  classification  based  on 
Medicare  data  with  that  obtained  from  records  abstracts,  especially  for  the  laparoscopic 
cholecystectomy  cases.  In  many  instances,  patients  classified  as  outpatients  spent  several  days  in 
the  hospital.  Since  this  study  had  the  advantage  of  being  able  to  compare  the  medical  record  finding 
with  the  billing  data,  some  opportunity  to  test  the  effectiveness  of  using  the  latter  was  afforded.  In 
general  the  MEDPAR  bills  worked  well,  but  there  were  omissions.  The  experience  with  Piart  B 
claims  was  less  encouraging.  In  some  cases,  billings  for  the  procedure  could  be  found  but  not  a 
bill  for  the  surgeon.  Outpatient  cases  might  appear  under  outpatient  claims  in  MEDPAR  or  in  Part 
B  files,  in  neither,  or  in  both. 
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